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Abstract. 

Code Division Multiple Access (CDMA) in which the spreading code 
assignment to users contains a random element has recently become a cornerstone 
of CDMA research. The random element in the construction is particular 
attractive as it provides robustness and flexibility in utilising multi-access 
channels, whilst not making significant sacrifices in terms of transmission power. 
Random codes are generated from some ensemble, here we consider the possibility 
of combining two standard paradigms, sparsely and densely spread codes, in a 
single composite code ensemble. The composite code analysis includes a replica 
symmetric calculation of performance in the large system limit, and investigation 
of finite systems through a composite belief propagation algorithm. A variety of 
codes are examined with a focus on the high multi-access interference regime. In 
both the large size limit and finite systems we demonstrate scenarios in which 
the composite code has typical performance exceeding sparse and dense codes at 
equivalent signal to noise ratio. 
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1. Introduction 

Code Division Multiple Access (CDMA) is an efficient method of bandwidth allocation, 
employed in many to one wireless communication channels [1]. Schematically, each 
user is allocated a code by which to modulate some sent symbol across the bandwidth. 
The signal arriving at a receiver (base station) is a superposition of the user signals and 
channel noise; with carefully chosen codes, the source signals sent by each user may be 
robustly inferred. The problem we wish to address is one of multi-user detection, in 
which the bandwidth access patterns for different users are random and not correlated 
in such a way as to prevent, or reduce optimally, Multi- Access Interference (MAI). 

The base station must extract information from the relevant parts of the 
bandwidth in order to decode for a particular user. It is convenient to consider two 
spreading paradigms. In the first, each user transmits on the full bandwidth allocating 
a small amount of power to each section. Alternatively, the user may have power 
concentrated on one or several small sections of the bandwidth. In the former case we 
say the code is dense, and in the latter, sparse. We will consider the case where the 
access patterns are random and uncoordinated between users [H [3l [H |5l [6] . We also 
consider one simple case in which there is coordination between users. Coordination 
between the users allows opportunities to reduce MAI, thus producing an improved 
performance. 

In the CDMA with dense spreading patterns, bits of information may be 
transmitted at a near optimal rate using pseudo-random dense spreading codes [T], 
which are amongst the best understood CDMA methods. These codes are generated 
independently for each user, and may be quickly decoded by a matched filter or 
modified message passing methods under standard operating conditions. A more 
recent interest has been in the sparse analogue of these codes, in which performance 
is comparable but decoding is based upon Belief Propagation (BP) [6J. There exists 
enough latitude in parameters and channel properties encountered in real systems to 
anticipate that each method may be optimal in different applications and operating 
conditions. 

We argue that a principled simultaneous use of sparse and dense codes might 
provide some useful diversity, and it is this possibility which is investigated here. In 
particular, we consider a linear combination of sparse and dense random codes to 
study their generic properties. 

We are also interested in the case of composite systems more broadly than for the 
purpose of coding multi-access channels [3 [8] . In statistical physics sparse random 
graphs (representing sparse random codes) and fully connected graphs (representing 
dense random codes) form the basis for many statistical models. These models allow an 
analytical appreciation of phenomena in physics and more general complex systems [9] . 
The CDMA model provides some interesting indications of new phenomena when both 
sparse and dense processes are at work. 

1.0.1. The model The model for wireless multi- user detection is idealised as a linear 
vector multi-access channel subject to Additive White Gaussian Noise (AWGN). The 
transmission between each user and the base station is perfectly synchronised and 
power controlled. In other words, the coding is received as intended at the detector, 
there is no fading or scattering of the source signals. 

The bandwidth is discretized as N time- frequency blocks (chips), so that a 
vector describes the spreading pattern across the bandwidth. Each users (labelled 
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Figure 1. During a bit interval each user k sends a bit bj. modulated across 
some set of chips at constant amplitude, either phase or anti-phase (a 2 symbol 
alphabet for each user). The signal arriving at the base station is a superposition 
of user signals and channel noise ui. The signal is detected and decoded to recover 
estimates of the sent bits. In this diagram the number of users K = Z and number 
of chips N = 5, though in this paper we consider cases where N and K are large. 
We omit the fading factors for brevity from this description F^i^ = 1. 



by = 1 . . . K) is assigned a modulating code (s^) for transmission/detection of a 
random bit, bk = ±1 sent to/from a base station. The channel load is x = K/N, 
which is finite. Consider the transmission case where the base station has knowledge 
of all codes in use. A superposition of the user transmissions, along with noise (u;) 
arrives at the base station as shown in figure [1] Assuming perfect synchronization of 
the user transmissions with the chips the received signal on each chip ^ is 

y,i = (^t^ + ^bkFf^kS^k ■ (1) 

k 

where i^^^ is the user specific fading, which may, in practical applications, be different 
for each chip and user. In order to allow good decoding the base station may 
coordinates the amplitude of codes so that in expectation the received Signal to Noise 
Ratio (SNR) is uniform for all users, which is a special case of power control. For 
example, users at greater distances (suffering greater fading) in a real cellular phone 
network will be instructed to use a higher transmission power to mitigate this effect. 
We assume such a determination of relative power levels has been achieved, so that 
we may take Fk^j, = 1 and normalized codes s^s^ — 1, the normalization can be taken 
without loss of generality in the analysis since the noise power spectral density can be 
rescaled accordingly. If the noise distribution is independent and Gaussian distributed 
on each chip then the SNR per bit is identical for all chips and defined as 

SNRb = l/{2al) , (2) 

where (t§ is the variance of the noise per chip. Note that the total signal to noise ratio 
(the power spectral density) contains an extra factor of x- 

1.0.2. Sparse, dense and composite spreading codes In standard dense CDMA a 
code is assigned to each user so that on any chip the signal transmitted is modulated 
according to s^f., which is non-zero for most or all chips. The Binary Phase Shift 
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Keying (BPSK) random ensemble takes for each user a normalized code sampled 
uniformly at randomly from {±1/\/N}^ . 

In one definition of sparse CDMA, by contrast, there is no transmission by user 
k except on some finite subset (dk) of Ck chips (/xi . . . nCk)- Let C be be the mean 
connectivity users in the ensemble, which is small and finite for the ensembles we study. 
The simplest case to consider is one in which the set dk is sampled independently and 
uniformly from the set of N choose C'k possible chip combinations for each user. 
Throughout this paper we consider the number of accesses for all users to be identical 
Ck = C. For the subset of C chips transmitted on by user k {fj, G dk), in the standard 
case, BPSK modulation is used so that s^^ is sampled uniformly from {±-y/l/C}'^. 

To assume the number of accesses per user is a random, sampled from a Poissonian 
distribution, would also seem a natural choice, but note that with this choice some 
users will fail to access the bandwidth altogether. These disconnected users would 
strictly limit performance, and constitute a fraction exp(— C) of all users. Choosing 
the uniform distribution allows uniform access to the bandwidth for all users in 
expectation, and we expect this to be optimal in many senses 4 . 

In both cases the modulation method is BPSK. This is not the only viable 
modulation method, other simple modulation methods may result in improved 
performance [lOj . One convenient simplification in the case of BPSK is the symmetry 
of the modulation, which means that for purposes of analysis the transmitted bit 
sequence can be chosen as b = 1 (all I's) without loss of generality. 

The codes we shall consider in this paper are composite codes which may be 
constructed as a sum of a sparse and dense codes sampled according to the definitions 
of this section. The composite code is sketched in figure [21 it involves both a sparse 
and dense random codes with power normalized to 1. The power ratio between the 
sparse and dense parts is controlled by a parameter 7, the spreading code may be 
written in this case as 



If the sparse and dense codes are normalized the new code will be normalised up to 
a small {0{1/N)) factor which is not important in the large system analyses. In the 
finite systems we correct for this difference. 

The difference between composite and dense codes is in the hierarchical nature 
of the modulation sequences, all chips are transmitted on, but with two scales of 
transmission (provided 7 3> ^/N), so that in terms of information transmission and 
decoding the subset of chips dk for user k does not become insignificant as N becomes 
large, as would be the case for any subset of the chips given a dense code for user k. 

1.0.3. The case for random codes Random codes, sampled according to some 
ensemble description, offer flexibility in managing bandwidth access by allowing 
code assignment by independently sampling for each user, and also have robust self- 
averaging performance for large system sizes. Furthermore the unstructured nature of 
codes makes them useful in adversarial models of communication, or in the presence 
of structured noise effects. 

Random codes, sampled independently for each user, interfere in the channel. 
Optimal encoding of sources would involve a correlation of codes so as to minimise 
MAI. It has been shown that standard dense and sparse random codes can achieve a 
BER comparable to optimal transmission methods in the AWGN vector channel with 
only a modest increase in power. Optimal is here by comparison with transmission in 
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the absence of MAI, the single user case, with comparable energy per bit transmitted. 
The small increase in power required to equalise performance is often a tolerable 
feature of wireless communication. 

CDMA methods can be formulated so as to reduce or remove MAI; for example 
orthogonal codes {{sD^sl, = 6k,k') can be chosen for sparse and dense systems, 
whenever x ^ 1j achieving a single user channel performance. Gold codes [111 are 
the structured generalization of this beyond x = 1 in the case of the dense ensemble, 
where the distance between codewords is maximized, but some MAI is present. A 
sparse version of the orthogonal case is Time or Frequency Division Multiple Access 
(TDMA/FDMA), whereby each chip is accessed by at most a single user. For x > 1 
(but sparse, C — 0{1)) improvement is also possible beyond the random ensemble - 
for example by enforcing the chip connectivity to be regular [4]. 

In many cases only limited coordination of codes might be possible, so that MAI 
is an essential and irremovable feature. Random code ensembles are used in this paper; 
with random MAI present implicitly. The composite code ensemble, like sparse and 
dense cases, has a structure which is suitable to detailed mean-field type analysis in 
the spread-spectrum limit, and as will be shown, can outperform sparse and dense 
analogues in some reasonable parameterisations of the linear vector channel. 

The random coding models, with a little elaboration, may also approximated 
different scenarios other than ones corresponding to deliberately engineered codes. 
Consider for example a TDMA code, which is a sparse orthogonal coding method, so 
that the transmitted signal for any user is intended for a unique chip (time slot). In a 
practical environment this code may not arrive perfectly but might have a significant 
power component delayed by random processes, which contributes to unintended chips. 
This may occur in practice by way of multi-path effects [12j . In terms of the optimal 
detection performance, the properties may more closely resemble sparse CDMA, rather 
than a MAI-free TDMA method. If the paths are more strongly scattered across a 
significant fraction of the bandwidth a random dense inference problem is implied. 
Finally, a scenario with a few strong paths and many weak paths may apply, then the 
detection problem might be best approached as something similar to the composite 
code model. 

1.0.4- The case for a statistical treatment The large N (spread-spectrum) scenario 
is an efficient multi-user transmission regime 13 , and one in which we expect typical 
case performance of different codes drawn from our ensemble to converge. In the large 
system limit N, K oo the typical value for performance statistics, that describe 
accurately almost all samples, are the quantities of interest. Taking C finite the 
properties of dense, sparse and composite random codes are distinguishable and may 
be calculated from a free energy density. The spread-spectrum and many user limit 
is a standard benchmark, and statistical mechanics methods are established tools 
in analysis of such cases [14]. Often systems of practical size reflect strongly the 
properties inferred from the large system result; however, finite size effects may be 
critical in some practical issues of decoding. 



Composite CDMA - A statistical mechanics analysis 



,(1-Y)N" 



Dense 



YC" 



Spaise 



Composite 



Figure 2. (colour online) The upper figure shows a standard random BPSK 
code for a dense system. The middle figure shows the sparse ensemble where all 
power is concentrated on a few [C = 3) chips at higher power on each chip. The 
composite system is a superposition of these systems, the power in the sparse 
system is normalized to 7 and in the dense code to 1 — 7. For the finite size 
examples we avoid the collision between the two codes on C/N links by setting 
the dense code to zero on the set dt: and redistributing the power uniformly. In 
the large system limit the overlap is in any case a negligible effect. 



2. Method 

2.1. Calculation of Optimal Performance 

To determine optimal performance we analyze the mutual information between the 
sent bits and received signal per bit I{b,y;{s}). In the large bandwidth limit 
and for typical bit sequences, channel noise and code samples performance will be 
indistinguishable from the mean quantity 

se= hm l(-P(6,y)log2(P(6,j/))) (4) 
K-ioo K \ I b,y.{s} 

Where for our ensemble we will take 6=1. This quantity is expected to be self- 
averaging with respect to typical instances of the code. Self-averaging assumes that 
cases differing significantly from the mean value become statistically negligible as 
K — xN becomes large. The mean of the mutual information with respect to these 
quantities is then a sufficient measure of equilibrium properties. 

This quantity may be most concisely determined through a statistical mechanics 
methodology. The probability distribution should be separated into a noise part, 
which is marginalised over, and a bit estimate part, which is the part of interest 

/ K \ 



p{b,y)^p{b) J n 



=1 



(5) 



For decoding purposes we assume an AWGN model, with signal to noise ratio per bit 
estimated as [3Q (Q abbreviating SNR in equations), and marginalize with respect 
to this distribution to find a quadratic form in the exponent. The assumed form is 
within our model system correct, but in more complicated systems the form may still 
be used as an estimate. The prior is on bits is assumed to be uniform. The analysis 
then becomes equivalent to evaluation of an Ising spin model with Hamiltonian 

N / K \ 2 

H(T) = Q^U^-^S^feTj , (6) 
Ai=l V k=l I 

and inverse temperature (5. The dynamical variables {t^} approximate the random 
variables using the evidence and the code § (quenched variables). The mutual 
information is affine to the free energy density for this model and takes an upper 
bound of 1 bit. 
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The free energy niay be evaluated through the rephca method 9 . The self 
averaging free energy, averaged over code samples and instances of the channel noise, 
may be calculated by use of the replica identity 

Pf= lim logZ)= lim -— lim^ ; Z = V exp -/Jfffr) . (7) 



To apply the method we take n as an integer, with becoming a product of partition 
functions in different dynamical variables, but shared quenched variables [15] . For each 
of the replicated partition functions a Gaussian identity may be applied to reduce the 
square in the exponent to a linear form 



exp{-/3H(T,")} = n / exphA^ /2] exp ^J~^\c, 




(8) 



Representing as a product of the quenched variables h = l,{s} and a; and then 
separating those parts of order 1 / ViV in the exponent (due to the dense code) , we 
have 



^/7E 



(9) 



The sparse and dense code parts are now factorized in the exponent and the 
quenched averages may be made independently according to standard sparse and 
dense methodologies [H [2j [10] ; introducing order parameters for the sparse and dense 
induced correlations independently we have the free energy in a variational form (jlOp . 
The energetic (/3 dependent) and entropic parts are separated for clarity, and the 
entropy has been assumed extensive. Assuming a Replica Symmetric (RS) form for 
the order parameters the free energy is determined by an extremisation 



/ =Extr 



WW ,Q,Q,m,m 



/e(/3) 



QQ 
2 



+ mm + / dW{x)dW{u)log 



1 + tanh(M) tanh(a;) 



(10) 



(11) 




2 cosh J2 



QX 



fe = - ^log 



Y\2cosh{uc) 

L 

llidWixi)] 



(12) 



1=1 




6(1 




The order and conjugate (hatted) parameters are W and W, normalized distributions 
over real fields, and Q,Q,m,,rh, which are scalars. The averages are with respect to: 
P{L), a Poissonian distribution (the excess chip connectivity distribution of the sparse 
sub-code), P{{£,i}), a uniform distribution on {±1}^ (the modulation sequence), and 
P(A), a normally distributed variable. Parameters 

A -aoV/3 + x(l-7)(l-Q) , 



A'-- 



+ x(l-7)(l-2m + Q) 
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describe the assumed and correct signal to interference [channel noise and dense sub- 
code interference] ratios, respectively. If (3 is not taken as the Nishimori temperature, 
/3 = 1, then A ^ A!. 

The sparse code requires a full distribution to describe the contribution to the 
free energy, whereas the dense part requires only a few scalar parameters. Taking the 
limiting cases of large or small 7 one set of order parameters becomes negligible and 
the usual sparse/dense expressions are recovered [11 [2]. The complexity of evaluating 
the free energy is limited primarily by the sparse part. 

At the Nishimori temperature, corresponding to the correct detection model 
(/? — 1), the free energy is correctly described by the RS assumption [l6j. With this 
assumption we can make use of a symmetry to reduce the number of order parameters 
in the dense part (Q = m,(5 = ™)- Extremization of the free energy leads to the 
following sets of saddle-point equations in the order parameters. 



(13) 

Q = (/nti^«cW^K)tanh2 (E^^^,., + g 




Q = x(i-7)(i-Q) 

where the distributions by which averages are taken are unchanged. The quantity Z' 
is a local mean field partition sum, only the dependence on one summation variable 
is written explicitly. 



^'(To)=n 



i=\ 




t^<--\{^*t^^-n)\ \ . (14) 



Equations are easily modified to include the general (5 case. 

From the free energy at the saddle-point, by application of small conjugate 
field against the terms i^) '''n • • ■'''11,1 is possible to identify P{H) with the 

distribution of log-posterior ratios of source reconstruction in typical instances of the 
quenched model. Let 

Hk - ilog(P(6fc = l|S,y)/P(6fc = -l|§,y)) (15) 
then the quantity 

K 

P{H) = lim — V 5{H - Hk) = / dudhW{u)W{h)8{H - (u + h)) . (16) 

fc=l '' 

once an analytic continuation is taken in the sum. From this observation the bit error 
rate is determined by the integral 



BER === / P{H)dH . 

J —OC 



(17) 



The mutual information, with regards to typical case of ([5]), is attained by an affine 
transformation of the free energy. The entropy is also calculated from the same 
treatment, at the Nishimori temperature the energy is l/(2x). 
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2.2. Decoding: multistage detection and BP 



The idealized achievable performance is calculated in the limit of large N under the 
RS assumption. In practice one must deal with finite systems, and the finite size 
effects tend to degrade performance relative to the ideal. However, for reasonable 
size systems {N > 100) and 7 » the properties of composite codes in decoding, 
based on suitably constructed heuristics, become distinguishable from the performance 
through sparse or dense decoding methods, and approach in many cases the solutions 
predicted by the replica method. 

We consider two algorithms, the standard multistage detection (MSD) [1], based 
on iteration of a vector approximation to the sent bits 



6*+i=sign 



(skfy-J^Ykk'bl, (18) 

k'\k 

and a modified form of BP. MSD is a heuristic method [1], which works well in 
dense codes and simple noise models, provided MAI is not too large. BP is based on 
passing of conditional probabilities (real valued messages) between nodes in a graphical 
representation of the problem (I7j (a schematic description of the method appears in 
figure [3]). 

The most time consuming step in BP is a trace over the states attached to a 
particular factor node in order to determine the evidential message, a naive approach 
in the dense case requires 0{2^) floating point operations. However, due to the 
central limit theorem the dependence on the weakly interacting bits, not connected 
strongly through the sparse code, is equivalent to a Gaussian random variable and 
the marginalization is replaced by an exact Gaussian integral. This reduces algorithm 
complexity asymptotically to O(iV^). 

The approximation leads to a more concise form for the evidential messages 
(passed from factor nodes to variable nodes): 



Zf,^k{Tk)= Yl 



ied^\k L n 



^exp{/3/iJ_^Ti} 



(19) 



(20) 



A* 



X exp ■ 



s^/t; - ^ s^,i tanh(/3i7/) 



i\a^ 



K 



(21) 



x(i-7) i-^Eta^i^'(/3i/n)(?2) 



- J2 slits.nh\(3hU^ 

i\{k,a^} \ ' 1=1 

where a further simplification is possible for messages passed along dense links, using 
an expansion to leading order in s^^ = 0{1/^/N), 



1 



PA 



J2 tanh(/3i/*) - J2 «Mfe tanh(/3/iJ__^) 

i\{kM^} led^. 



, (23) 



as constructed in [18] . In these expressions the notation = indicates those equations 
where some 0{1/N) corrections have been eliminated, the most critical being the 
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replacement of the full marginalization over densely connected variables in (I2ip by 
a Gaussian integral that is taken analytically. At termination time the set of bits is 
determined by 6 = sign Evidential messages may be combined in a standard 

way to give marginal log-posterior estimates for the source bits 

H-*+i ^ i„ . Pjbk = My) t (r,A\ 

=2^1°^P(6. = -l|y)=^^/-- (2^) 

and variable messages (passed from variable nodes to factor nodes) 

hl^-}, = Hi+'-ul^,.. (25) 

The algorithm remains 0{N'^), and is less convenient than (fTS]) : however, the 
expression may be manipulated without introducing any additional errors at leading 
order in N to an algorithm without dense messages [lOj . the manipulation is an 
application of that proposed in [THj. The asymptotic complexity of the algorithm 
is comparable to MSD and other good algorithms, although this disguises a large 
constant factor and some structures which might be difficult to implement efficiently 
in hardware. 

Composite BP is applied as a heuristic algorithm based on an unbiased 
initialisation of the messages, in the hope that the various simplifications on the 
algorithm do not produce strong finite size effects. BP exactly describes the marginal 
probability distributions only if the messages , , ([^5]) converge to a unique fixed 
point, since in this case H describe the log-posterior ratios. There are two scenarios 
to be concerned about, either the BP messages fail to converge, or they converge 
to an incorrect fixed point - both scenarios occur in different decoding regimes for 
CDMA. The requirements for standard BP to successful decode are closely related to 
the assumption of RS, hence the similarity of the minimization process for W, W (|13p 
and the BP equations. 

In studying systems of finite size we consider two message update schemes for both 
MSD and BP. The first is a parallel update scheme where all variables are updated 
such that the values of the current generation of messages (t -I- 1) are conditionally 
independent given the previous generation of messages t. The second schemes is a 
random stochastic update method, the updates are applied to all messages in the 
population, but in a random order. As soon as a variable is updated it is made 
available to subsequent updates, the messages in a single generation (t+l) are then not 
conditionally independent given the previous generation (t). The sequential update 
method is slower to implement, but helps to suppress oscillations observed in some 
parallel update schemes, that lead to unphysical solutions. 

A measure of convergence for the BP is the mean square change in variable states 

^ti^l-Ht'^y , (26) 



i=l 



whereas the convergence measure for MSD is similar but with the log-likelihood ratios 
replaced by the bit estimates. An exponential decay in this quantity, or an evaluation 
to zero, would be characteristic of a convergent, or converged, iterative method. 



2. 3. Properties of decoders in finite systems 



MSD is an iterative method which works very well in systems with small load x 
and mixing parameter 7. In the first iteration the achieved result is equivalent to a 
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matched filter. In subsequent iterations tlie estimates are updated, but because the 
information is rather crudely used the consequence can be instability of the iterative 
procedure. For instance, the examples shown later reveal very poor performance of 
MSD when MAI is large. Since MSD is based on filtering it is not so successful for 
composite ensembles as for dense ones, and its reliability in dense codes improves as 
system size increases. 

The critical scenario in which BP is guaranteed to produce the correct marginal 
posteriors is that the graphical model is tree like (see figure |3]). However, BP often 
produces a reasonable performance in loopy models including sparse [B] and dense [TH] 
CDMA. The failing regime in BP corresponds to large x i^i the sparse and dense 
codes, but at intermediate 7 the instabilities appear to be worse, or new instabilities 
are apparent. 

In many of the cases studied we found that BP did converge in the marginal 
log-likelihood ratios, this was the case for systems at small Xi and/or high SNR. In 
other cases the fields did not converge, and instead a steady state was reached at 
the macroscopic level - remembering the BP equations describe a dynamic algorithm, 
which does not obeying detailed balance, this might be expected. The steady state 
is one in which the distribution of messages converges up to finite size effects, but 
the individual messages do not converge. Steady states were characteristic of systems 
initiated with random or unbiased messages at high x- The estimates determined from 
the distribution of messages in the steady state typically corresponds to a high BER 
estimates. 

In regimes where message passing is unstable the detectors may still be used to 
provide an estimated, subject to some termination criteria. Variations on MSD and 
BP involving heuristic tricks may avoid some of these effects, but some of the standard 
methods may be unsuitable to the composite model. Experimenation with the update 
scheme demonstrated improved results in MSD for example. 

The dynamics of numerically solving the saddle-point equations (|13|) are very 
closely related to the iteration of BP. The dynamics of the iterations (figure [5|) appear 
smooth and systematic even at large x based on a numerical solution involving 10000 
points. However, in addition to the use of a large system size, control was exercised 
over finite size effects through selective sampling of quenched variables in the update 
step ([T9l) , which is not possible in BP. Therefore any realisation of the problem in BP, 
even at an equivalent system size, is not expected to produce such uniform effects. 
Many qualitative features such as the speed of convergence appear to be reproduced 
in some of our finite realisations. 

Naturally we expect there to be finite size effects, the trends presented appeared 
consistent across a range of system size from O(IOO) to 0(1000) chips. No structured 
attempt is made to calculate these effects, or to distinguish the contributions due to 
the different 0{1/N) approximations in the algorithm and other instabilities implicit 
to BP. Working with a sufficiently large graph is problematic since the algorithm is 
asymptotically 0{N^) rather than linear, and it is necessary to store and manipulate 
a K X N modulation pattern matrix. 

2.4- Replica symmetry and the phase space 

At the Nishimori temperature the RS solution is guaranteed to describe correctly the 
thermodynamically dominant states. Across a range of parameters we find two saddle- 
point solutions: one corresponding to a locally stable bad solution (bad decoding 
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Factor Graph Belief Propagation 

Transmitted bit estimates 




Figure 3. (colour online) Left figure: The factor graph representation of the 
problem is useful, especially in visualizing the topology of sparse problems. Right 
figure: BP based on updates of link variables is often successful in determining a 
probability distribution consistent with the evidence, for both sparse and dense 
systems. The messages passed are represented by arrows and are conditional 
probabilities in BP, other message passing schemes are also possible, a special 
case being multi-stage detection. 



performance) and one to a locally stable good solution (good decoding performance). 
The metastable solution (solution producing the higher free energy) is irrelevant in 
determining the equilibrium properties since it does not contribute to the free energy 
at order N. However, in terms of dynamics or local sampling the bad solution can 
be dominant even where it does not describe the equilibrium phase. We find these 
metastable regimes at intermediate SNR and high MAI (above x = 1-5 [1]). 

The RS solution obtained at equilibrium appears sufficient to describe the good 
equilibrium and metastable solutions. These are the local solutions to the free energy 
that correspond to states clustered about the encoded bit sequence b. In this case 
the phase is connected in state space, and so we expect the dynamics of the system 
to be relatively simple, so that the phase space can be explored by local sampling 
methods such monte-carlo. BP will be locally stable in the vicinity of this solution, 
in the absence of competing local minima we can expect convergence towards this in 
specific realisations. This solution to the free energy exists when SNR is sufficiently 
large. 

By contrast we expect there to also be a bad (liquid/paramagnetic) equilibrium 
solution when SNR is small. The field term in the Hamiltonian means the 
magnetisation is never zero, but we expect there to be a suboptimal ferromagnetic 
solution which is also connected in state space, and that has similar properties in 
terms of BP and sampling. The good and bad equilibrium solutions are guaranteed 
to have simple phase space structures described by RS, this conclusion and some 
thermodynamic properties can be calculated without the replica trick or cavity 
method [16^. 

Finally we can anticipate that at high MAI and intermediate noise there may be 
a bad metastable solution. The bad metastable solution emerges continuously from 
the bad equilibrium solution with increasing SNR and so will be characterised by 
a connected phase space for some parameters. However, as the noise decreases we 
might expect this solution to become fragmented and the RS metastable solution to 
become unstable, but this instability is not tested. An indication of the failure of RS 
is the negative entropy in some metastable solutions, which is not viable. It is not 
uncommon for systems with simple connected phase spaces to exhibit negative entropy 
when the RS ansatz is applied under the assumption of extensive entropy [20 , as we 
employ in the calculation. A result without negative entropy can be formulated by a 
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minor variation on the RS approach caUed frozen rephca symmetry breaking (RSB), 
which effectively rescales the temperature. However, it is not certain that this solution 
will be correct without a local stability analysis towards other forms of RSB, in many 
other systems negative entropy is one indicator of a failure of the connected phase 
space assumption [15]. 

In the bad metastable state, and also in the bad equilibrium phase away from 
the Nishimori temperature (/3 > 1), the connected description is possibly incorrect 
even where the entropy is positive; an RSB formalism may be applicable. The good 
solution is likely to be well described by RS at all temperatures, since it is a state 
clustered around the encoded bit sequence. For /? > 1 the RS approximation produces 
a variational approximation to the thermodynamic behavior. The RS approximation 
can also describe exactly the metastable states in some regimes, but this is not the 
case in general. 

The hypothesis of a connected state described by the RS treatment has 
consequences for dynamics, as do the various hypotheses on the nature of RSB, 
should it occur either in a search for the ground state [21], or at some intermediate 
temperature. On specific realizations of graphs we might expect BP to converge for 
the states correctly described by RS. However, in finite graphs we might expect strong 
finite size effects to dominant behaviour, so that in the absence of a scaling analysis 
conclusions cannot be drawn directly from simulation results. In cases where BP is 
unstable due to RSB or finite size effects we might nevertheless expect to uncover a 
steady state of the dynamics that is strongly correlated with nearly optimal solutions 
and is useful for extracting an estimate of sent bits. 

3. Results 

3.1. Parameters considered 

The model constructed is already quite simple, avoiding some of the practicalities of 
real channels and making no attempt to optimize for finite size effects in the composite 
ensembles. However, even with these simplifications the channel produces interesting 
behavior. In order to demonstrate the equilibrium properties of composite codes in 
such a way as to produce strong contrast between the ensembles we work with systems 
with C — 3 and x between 3/5 and 2. Beyond this range of x results appear by 
experimentation to be very much a continuation of the trends highlighted. 

Analysis of the sparse code (7 = 1) is for this range of parameters a loopy inference 
problem, but is sufficiently far from the percolation transition to be well behaved. At 
the same time C = 3 is sufficiently small to allow quick decoding and produce a 
contrast with the dense code. It has been noticed since the early days of studying 
sparse codes that the mean connectivity of the sparse code ensemble C does not 
have to be very large at all for results to become indistinguishable from the dense 
code dlH]. 

A lower bound to the achievable bit error rate in all ensembles is given by the 
single user Gaussian channel (SUG) result over a bit interval 



which is the complementary error function of SNR. In the absence of MAI this lower 
bound can be achieved if spreading patterns are coordinated so as to be orthogonal. 




(27) 
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On the vector channel this orthogonahty is possible only ii K < N, unavoidable MAI 
at higher loads will strictly degrade performance. 

The saddlepoint equations are solved by population dynamics f23J, 

an iterative method for solving the saddle-point equations using a histogram 
approximation of the distribution W (10000 points are sufficient to attain our results). 
Evolving the order parameters from initial conditions that correspond to low and high 
BER finds either the unique solution or a pair of locally stable solutions. 

3.2. Equilibrium behavior of unique saddle-point solutions 




Signal to noise ratio, SNR^ [decibels] 

Figure 4. (colour online) The figure demonstrates the BER determined from the 
order parameters at the equilibrium solution of the free energy for various SNR 
and X = t- The curves represent different ensembles (7), with the single user 
Gaussian (SUG) channel lower bound also displayed (dotted line) for comparison. 
Error bars are significantly smaller than symbol size for BER above 10"*, and 
are excluded for clarity. The lower bound is approached for the CDMA codes at 
large and small SNR, the dense code is best amongst the random codes. The code 
with an even power distribution between the sparse and dense parts (7 = 1/2) 
is not easily distinguishable in thermodynamic performance from the dense code, 
even where the spread of codes is greatest (inset). 

Generally with x ^ 1.5 there is a unique solution of the saddle-point equations 
with a smooth transition between bad and good solutions as SNR is increased. The 
population dynamics equations require very few iterations and results can be achieved 
with relatively fewer points in the histogram. The normal working range of CDMA is 
often by design one with a relatively small load (x < 1) and so falls into this class of 
behaviour. 

The trends in BER for the large system limit for x = 1 are demonstrated in 
figure m The dense code ensemble achieves a smaller bit error rate than the sparse 
code ensemble, and the composite code ensembles interpolate between these. With 
7 = 0.5 the curve is indistinguishable at this magnification from the dense curve, 
with balanced power in the two codes performance resembles the dense code. At 
intermediate SNR there is a large gap in BER, which narrows in the limits of high 
and small SNR. Trends in the free energy follow a similar monotonic pattern - the 
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Saddlepoint iterations 



Saddlepoint iterations 



Figure 5. (colour online) The dynamics of the order parameters determined by 
iteration of the saddlepoint equations is shown for x = 5/3 and SNRb = 6dB, with 
a large population of lO'' points to represent the distribution W il3l . Evolving the 
saddlepoints from either Ferromagnetic or Random Initial Conditions (FIC/RIC) 
discovers either the unique solution (7 = 1 or 8/9), or two locally stable solutions 
(7 = or 1/2). Left: The maximum free energy is determined by the system of 
maximum entropy at the Nishimori temperature. In cases of small 7 there are 
two candidate solutions. The fluctuations are visible in some curves and are due 
to the sampling method, these fluctuations are not sufficient to escape the local 
solutions in the cases of metastability. Right: BER is widely spread, for small 
7 the thermodynamic solution is the good solution in this example. At larger 
7 there is a unique solution which is intermediate between the metastable and 
thermodynamic solutions at small 7. 



The regime of high x is of greater theoretical interest in multi-user detection since 
this is where MAI causes results to differ substantially from single user models. As 
X is increased beyond 1.5 a spinodal point may be reached beyond which there are 
multiple locally stable solutions to the saddle-point equation. 

In regimes with a competition between locally stable attractors, or with one 
marginally stable attractor convergence of the saddlepoint equations is slowed down; 
one such scenario is shown in figure O At x = 5/3 there is a unique solution for 
some of the sparse and composite ensembles, but not for the dense ensemble. In 
this example the composite code solution is superior to the sparse solution, and the 
dense metastable (bad) solution. The best solution is the dense thermodynamic (good) 
solution. As shown in figures [6] and [5l the entropy is positive for all the thermodynamic 
solutions. However, at larger x ^^^d higher SNR the metastable solutions can have 
negative entropy, indicating an inadequacy in the RS description. 

The saddle-point solutions for our ensembles with load x = 2 at a range of SNR 
is shown in figure [HI For this load metastability is present at all 7 values. Where 
the solution is not unique the correct and metastable solutions can be distinguished 
from the free energy (equivalently entropy at the Nishimori temperature). At the 
Nishimori temperature there is a second order transition, the energy is equal to 1/2 
in both solutions, which is realized as a discontinuous transition in the BER. In the 
metastable regimes we find the entropy evolves towards a negative value as SNR 
increases, the correct metastable state in the negative entropy regime is described by 
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Signal lo Noise Ratio, SNF^ [decibels] Signal to Noise Ratio. SNR^^ [decibels] 

Figure 6. (colour online) The figure covers the same range of parameters as figure 
[4l but with a load x = 2. Two locally stable solutions are found by minimisation 
of the RS saddlepoint equations in a range of SNR for all 7. Left figure: The 
entropy indicates a second order transition between the good and bad solutions 
for each ensemble. At SNR greater than the thermodynamic transition point 
metastable solutions evolve towards a freezing point (s = 0) and a regime of 
negative entropy. The thermodynamic transition point is at significantly greater 
SNR in the sparse ensemble than the composite ensembles. The range of SNR 
for which metastability exists is minimised in composite systems with 7 fs; 8/9. 
Error bars are everywhere much smaller than symbol size. Right figure: The 
thermodynamic transition indicates a large gap between the saddlepoint solutions 
at the thermodynamic transition and spinodal points. The properties of the 
good and bad solutions change smoothly about the thermodynamic transition 
and freezing point. Right inset (a): The bad solution has high BER even at 
large SNR and becomes locally unstable at lower SNR for composite systems 
with 7 ^ 8/9. Right inset (b) Good solutions with larger 7 have lower BER and 
are stable at smaller SNR. 

the state space at the freezing point where entropy first becomes negative. 

Up to 7dB the bad solution is the thermodynamic solution in all ensembles. 
Close to the transition the best performing codes are composite ones with 7 ^ 8/9, 
but at lower SNR the regular code ensemble appears best. The composite systems 
displayed all have thermodynamic transitions near 7dB, the entropy and free energy 
of the sparse bad solution is much larger, so that thermodynamic transition does 
not occur until about 9.5dB. This entropy gap is a clear feature of the local freedom 
explicit in a sparse connectivity model and absent in the composite one. In the case 
of a regular sparse part, with a more homogeneous interaction structure, the gap in 
entropy and thermodynamic transition point are significantly reduced [3]. Amongst 
the good solutions, in contrast to the bad solutions, both the ensemble entropy and 
BER appear to be ordered by 7 for all SNR. 

The metastable solutions appear to be qualitatively similar in the composite 
ensemble to the sparse and dense ensembles [T9j |4]. What is interesting in the 
metastable regime is that the positioning of the composite state performance is not a 
simple interpolation between the sparse and dense ensemble results. In the example 
shown the metastable solutions for composite codes are at lower BER than either the 
sparse or dense metastable solutions. Furthermore, for 7 = 8/9 there is a unique 
solution beyond 8dB inspite of the persistence of metastable solutions in the sparse 
and dense ensembles at significantly larger SNR. 

We did not test the microscopic stability of the metastable solutions for the 
composite system, but this should be possible, in part, by a local stability analysis 
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of the RS description. It is expected that at, and above, the Nishimori temperature 
(/? < 1) the RS description wiU be locally stable even for the metastable states, as was 
found for the dense and sparse codes [12] [I]- 

The composite codes exhibit a thermodynamic behaviour most strongly 
contrasting with both sparse and dense codes when 7 < 1, and close to the 
thermodynamic transition of the dense code. The effect of distributing power mostly in 
the sparse code appears to destabilise the bad solution in some marginal cases. Results 
obtained by the replica method solved by population dynamics appear robust; only 
close to the dynamical transition points, where solutions are marginally stable, are 
not well represented. 

To understand the origins of this instability requires a more detailed investigation 
of the stability of the RS metastable solutions, and possibly an RSB type treatment. 
We suspect that the dense code behaves comparably to the external field at 
equilibrium, and this might be one way in which one could approach the problem. 

3.4- Decoder Performance 




# updates # updates # updates # updates 

Figure 7. (colour online) Mean BER (dashed line) and A (solid line) are shown 
for different ensembles, 7 = {0,1/2,8/9,1} from left to right, as a function of 
the number of variable estimate updates for BP and MSD implemented with 
parallel updates. SNRb = 6dB and x = 3/5 (A^ = 1000, K = 600): for 
each point 300 independent sparse and dense connectivity profiles were sampled 
and combined in proportion to 7, with channel noise randomly sampled from 
a Gaussian distribution. The convergence measure A I I26I I indicates exponential 
convergence in BP and non-convergence of MSD for all ensembles. The RS result is 
approached after 10 updates by the simulation average, but with some systematic 
error due to finite size effects. The MSD result does not improve beyond about 
five updates, but mean statistics reflect primarily the performance of the worst 
samples. 
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Figure 8. (colour online) The cumulative distribution function for the decoding 
at 7 = of the 300 samples taken, as in figure [7] is typical in structure of all 
composite systems. The BER found by BP has converged for all samples taken 
within 10 updates. The BER found by MSD continues to evolve between 10 and 
20 updates, with increasing BER for some subset of the samples. The median of 
the samples is close to the BER, but the cumulative distribution function is not 
yet approaching a tight Gaussian and finite size effects are thus important. Some 
percentage of samples obtain a zero bit error rate which accounts for a small part 
not represented on the logarithmic scale. 



In systems with x ^ 1 the equihbrium results are achievable by iteration of BP 
equations, this was established previously for the dense case in jl9j . Such an example is 
shown in Figure [7] with x = 3/5. The performance of MSD is poor, although initially 
the achieved bit error rate is improving with each iteration, over many iterations 
there is a clear oscillation. For systems of higher SNR and/or decreased x find 
the MSD result to be very close to BP and the theoretical result. The BP algorithms 
reproduce the equilibrium result to within a small error after only a few iterations, 
even in systems with only 600 users and 1000 chips (x = 3/5), across a range of 
7. Where unique saddle-point solutions were predicted by the equilibrium analysis 
decoding by BP normally produced a stable fixed point. The MSD results are not 
shown in subsequent figures but are suboptimal with respect to BP in all cases. 

A histogram of achieved BERs is demonstrated in figure [S) In the large system 
limit we would expect the cumulative distribution functions to converge towards step 
functions (self averaging) on the one of the thermodynamic solutions, but it is clear 
we are quite far from this scenario. BP converges quickly towards results of very low 
or zero BER. The MSD algorithm works very well, but more slowly than BP, for a 
subset of examples. In many other samples the performance deteriorates as MSD is 
iterated beyond the initial approximation (matched filter). 

If we consider a similarly sized system of 600 chips and 1000 users (x = 5/3) then 
we can observe decoding performance in a regime which should be characterized by 
metastability in the dense code but not in the sparse code. The final BER achieved 
in 300 samples for various systems is shown as a cumulative probability distribution 
in figure ini after 10 iterations and after 80 iterations. The sparse system is unimodal, 
with fast convergence in most systems. The dense ensemble is multimodal as expected, 
the convergence time towards the low BER solutions are very slow, and the majority 
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Figure 9. (colour online) For SNRt = 6 decibels and x = 5/3 (A^ = 600, K = 
1000) we demonstrate the decoding performance of some algorithms by presenting 
cumulative distribution functions in the BER based on 200 runs. The histograms 
from left to right represent mixing parameter values 7 = {0,1/2,8/9,1}. The 
sparse code samples (far right) converge in most cases after 10 iterations, and the 
median performance is close to the unique RS solution. The dense ensemble (far 
left) is after 10 iterations close to the median performance for the metastable RS 
solution (right RS soluton) . A subset of samples evolve further towards or beyond 
the thermodynamic RS solution (left RS solution). For 7 = 1/2, and 7 = 8/9, 
performance for most samples does not reach the asymptotic RS prediction for 
the metastable solution BER. 



of achieved solutions are close to the high BER metastable solution. Random initial 
conditions tend to produce steady states characterized by the bad solution, even if this 
corresponds to the metastable rather than equilibrium solution. The composite system 
equilibrium solution is unique for 7 = 8/9. For 7 = 8/9 some 40% of samples improve 
between iteration 10 and iteration 80, but 40% also worsen, the median performance 
is quite far from the equilibrium value. The equilibrium results for 7 — 0.5 are not 
closely approximated in the decoding experiments, the performance in BER is worse 
everywhere than the equilibrium prediction and the constituent dense and sparse 
systems. For large x it appears the finite size effects are more limiting in the case 
of the composite codes, particularly at intermediate values of 7. 

The composite systems shown in figure |9] does not come close to the performance 
of even the bad solution in either the median or mean for this system size except for 
large or small 7. The ability of the composite algorithm |18j is limited in achieving 
the equilibrium result for intermediate 7 than standard methods for the sparse [6] 
and dense |19| ensembles for systems of this size. A quantitative comparison of the 
equilibrium and finite size systems in the metastable regime with bulk statistics such 
as the mean is difficult due to the multimodal nature of the distributions. 

The decoder performance for systems of size 0(1000) seem to provide mean values 
for the BER, which are quite far from the theoretical values and unable to realise the 
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advantages of some composite codes predicted by the equilibrium analysis. There are 
many approximations made at 0{1/N) in construction of the BP algorithm [12] |18j . 
which are both systematic and random, and it is possible these are at the root 
of the BP instability for intermediate values of 7. However, it is noteworthy that 
even without elimination of dense messages, performance with the proposed update 
schemes are poor at intermediate 7. The ordering of updates may also be important, 
and other sensible schemes might be consider. For example to iterate only the 
sparse messages until convergence (a fast process) between updates of dense message 
dependent quantities. 



3.5. Regular sparse ensembles 




Figure 10. (colour online) Shown is the optimal performance for 7 = 
{0, 1/2, 8/9, l},x = li with a chip regular sparse ensemble as a component in 
the composite system. At high SNR the performance decreases with 7, at low 
SNR the performance increases with 7. For a small range of inclusive of the inset 
range, the composite codes outperform both the sparse and dense codes. 



A random ensemble with regular chip connectivity ,4] as well as regular user 
connectivity will be examined. There is not a strong case for use of these codes in 
practical scenarios, but they offer the possibility to demonstrate a positive result for 
the composite system which is reproducible in small system size experiments. 

An interesting alternative composite system to the user regular ensemble is one 
in which the number of accesses per chip (call this L) is equal for all chips. This 
requires global coordination of all users, with the loosening of this restriction neither 
the doubly-regular ensemble presented nor the random codes would represent the 
most high performing options. However, the regular code is interesting because it 
shares many of the topological features of the sparse random ensemble, but has very 
slightly lower MAI, reduced by a factor {L — 1)/L [24] over the random sparse or 
dense ensemble, and fewer finite size effects in the BP algorithm. The reduced MAI 
has the effect that at low values for SNR the unique stable solutions for the doubly 
regular ensemble is superior in BER to the dense ensemble. With this ensemble one 
can demonstrate a statistically significant result, in decoding by BP, for which the 
composite code ensemble outperforms the corresponding sparse and dense parts in 
BER. 
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The equilibrium behaviour of the doubly regular sparse ensemble was analysed 
in [3]. Replacing the standard sparse code ensemble by the composite one we can 
repeat the replica analysis and use the same set of BP equations (identical at leading 
order). In the analysis we find that the composite code performance interpolates 
the sparse and dense performance in low and high SNR regimes. However, in an 
intermediate range of SNR, where the bit error rate is equal in the sparse and dense 
models, the unique solutions for the composite ensembles become superior to both the 
dense and sparse ensemble result. The performance of several ensembles is shown in 
figure [ini 

Working with a simulation of 1000 users and 1000 chips we are able to demonstrate 
that the mean performance of several composite codes exceed the performance of 
there constituent parts, as shown in figure [TT] The results for 7 — {0,8/9,1} 
ensembles are close to the large system limit prediction, to within error bars. The 
composite code achieves the lowest bit error rate in expectation amongst the codes, 
and has convergence properties somewhere between the two extremal ensembles. As 
in previous experiments the performance for j — 1/2 is much poorer than the large 
system limit prediction. 

As can be seen the dense code fields are initially converging in a similar way to 
figure [71 However, at later time the fields appear to be becoming unstable again (at 
least for a subset of simulations that dominate the measure). This instability might 
be a direct consequence of the inaccuracy of the Gaussian BP approximation pT|) 
when BER in decoding becomes very small. Similar trends are seen in some of the 
composite codes, often the messages do not converge exactly but only to with some 
fixed variability. 

4. Discussion 

The equilibrium analysis demonstrates that in regions of metastability the composite 
coding structure, comprising of a sparse and densely connected component, might 
have some interesting and valuable properties. When power is approximately equal in 
the two parts performance is very close to the dense ensemble, but with only a small 
amount of power in the dense code properties are strongly distinguishable. At the same 
time, we have shown that in reasonably sized simulations our algorithmic approaches 
based on 0(1/N) approximations in the dense code prove to work relatively poorly 
in the composite codes. This instability in some composite codes can persist even 
in scenarios where the equilibrium analysis predicts a regime without a metastable 
solution, which would be the natural candidate for a non-equilibrium attractor. 

We predict the failure of the composite BP algorithm to be in part related 
to the Gaussian approximation in marginalization over states pi|) . which may be 
a poor approximation when messages become strongly biased. If this is the case 
then the problem may be avoided or mitigated by standard algorithmic tricks such 
as annealing or damping. When the messages become very biased, replacing the 
full marginalization by one considering only a truncated set of states might be a 
viable polynomial time alternative to using the analytical Gaussian approach. In 
small realisations of composite systems many heuristics might be employed. 

In the final section we presented results with a different type of sparse ensemble 
in which there was some coordination between users. Similarly there would be some 
value in considering either an ordered (optimised) sparse code combined with a random 
dense code or vice- versa. The ordered sparse code might provide a method for decoding 
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Figure 11. (colour online) At SNRj, = 4.5,x = li we show the mean results 
of 500 decoding experiments using 7 = {0, 1/2,8/9, 1} (from left to right) with 
a chip regular sparse component in each composite system. The BP equations 
converge except 7 = 0, where some samples were unstable. A similar effect is 
manifested in the 7 = ^ ensemble after about 45 iterations, but not within the 
scale of the figure. Each set of samples produced a BER in decoding close to the 
RS prediction except for 7 = 1/2, where decoding performance was substantially 
poorer. The BER by the RS result and sampling is best amongst ensembles with 
7 « 8/9. 



under ideal channel conditions, whereas the dense code provides a contingency and 
sonic of the advantages of the spread spectrum approach. At least in so far as a 
practical method this might be a well-motivated scenario. 

The composite code presents an interesting dichotomy in its surpression of the 
metastable behaviour, but greater apparent instability in simulation. Aside from 
standard convergence measures in simulations we have not thought of a concrete way 
to probe the origins of this instability in simulations. One possibility would be to study 
the equilibrium properties of the metastable state in more detail in the thermodynamic 
framework. 

A finite size scaling of the algorithm results would be valuable, unfortunately 
the need to manipulate an by if dense spreading matrix restricts us somewhat in 
moving to larger scales. The scales we have presented, and error bounds, are presented 
in a somewhat ad-hoc way but this is only to demonstrate the breadth of behavior. 
Many results in the cited papers go much further in dealing with the question of finite 
size effects in sparse and dense systems. 

We have developed a method which is applicable where the sparse ensemble 
includes graphs which are fully connected. If we consider a composite code with a 
sparse part below the percolation threshold a more fruitful analysis might consist in 
a two level analysis. By considering the different classes of disconnected components 
(within the sparse code) as the microscopic states connected through a homogeneous 
(dense code) interaction. This may also be the basis for better algorithms. 
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